Data streams v2 - #1192
Open
1egoman wants to merge 104 commits into
Open
Conversation
Contributor
Changeset ✓This PR includes a changeset covering all affected packages:
|
1egoman
commented
Jun 26, 2026
ladvoc
reviewed
Jun 26, 2026
ladvoc
reviewed
Jun 26, 2026
ladvoc
reviewed
Jun 26, 2026
1 task
1egoman
force-pushed
the
data-streams-v2
branch
3 times, most recently
from
July 6, 2026 20:05
3bf1532 to
d9e9bb3
Compare
1egoman
commented
Jul 6, 2026
1egoman
marked this pull request as ready for review
July 6, 2026 21:15
1egoman
requested review from
MaxHeimbrock,
alan-george-lk,
lukasIO,
stephen-derosa and
xianshijing-lk
as code owners
July 6, 2026 21:15
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
1egoman
marked this pull request as draft
July 8, 2026 19:22
1egoman
removed request for
MaxHeimbrock,
alan-george-lk,
lukasIO,
stephen-derosa and
xianshijing-lk
July 8, 2026 19:23
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
1egoman
added a commit
that referenced
this pull request
Jul 9, 2026
Breaks out a subset of the changes from #1192 into a pull request which exclusively moves the existing data streams implementation into its own crate, `livekit-data-streams`, and adds a new `livekit-common` crate (kept fairly minimal for now) for common utilities which `livekit-data-streams` and `livekit` both need. This is largely being scoped to a "lift and shift" type effort, more extensive refactoring of the `livekit-data-streams` crate will be done in #1192. ### Breaking changes There should not be any breaking changes from the perspective of `livekit`'s public api. If so, please surface them. ### Testing This pull request adds the relevant v1 data streams tests from #1192. More test coverage will be introduced as a follow up in the data streams v2 change. --------- Co-authored-by: Jacob Gelman <3182119+ladvoc@users.noreply.github.com>
…er compression mechanism was added in the future This should never happen but adding this extra check makes it just that much more bulletproof.
Test mode checks the contents of header.inline_content via is_inline, so .take() will clear this out and cause that field to be wrong on the receive end. However, this isn't currently being used in tests at all.
This is required for the python / node sdks to adopt the standardized data stream implementation, since it uses this feature for (I think) avatars
This allows configuring the data stream max payload size value when constructing a room.
The node and python sdks today have their own data streams implementations. If this branch were merged today, then it would mean that the new client_protocol=2 would be advertised, and these pre-existing implementations would receive data stream v2 messages, not v1 (ie, single packet data streams, compression, etc). So for now, add a flag which can be set in livekit-ffi to disable advertising the new client protocol so this can be merged. Once merged, node and python can be updated to pass this flag. Then, once data streams v2 is done on both node + python and both are updated, this flag can be removed by reverting this commit.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This was referenced Jul 23, 2026
ladvoc
approved these changes
Jul 27, 2026
| ); | ||
|
|
||
| if bytes_total.is_some_and(|total| total > self.max_payload_byte_length as u64) { | ||
| let _ = chunk_tx.send(Err(StreamError::PayloadTooLarge)); |
Contributor
There was a problem hiding this comment.
suggestion(non-blocking): Would it make sense to pass the total in the error variant for visibility?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This change introduces data streams v2, a set of major updates to data streams to add three things:
For information about important decisions made in data streams v2, see the spec. In particular, parts 1-5 provide a good high level overview of data streams v1 + v2, what these protocol improvements are aiming to acheive, and why certain choices were made when designing the protocol.
Also maybe of interest is the corresponding web sdk pull request.
These updates are fully backwards compatible - the sdk will fall back to data streams v1 whenever an old participant is in the room.
There is also a significant amount of new test coverage to data streams (both v1 and v2). There were some small bugs I found along the way which I fixed as well.